AutoML Leaderboard
| Best model | name | model_type | metric_type | metric_value | train_time |
|---|---|---|---|---|---|
| 1_DecisionTree | Decision Tree | logloss | 1.72988 | 2.69 | |
| 2_DecisionTree | Decision Tree | logloss | 4.04011 | 2.07 | |
| 3_DecisionTree | Decision Tree | logloss | 4.04011 | 2.1 | |
| 4_Linear | Linear | logloss | 1.96024 | 20.16 | |
| 14_LightGBM | LightGBM | logloss | 1.68199 | 2.69 | |
| 5_Xgboost | Xgboost | logloss | 1.70127 | 3.23 | |
| 5_Xgboost_categorical_mix | Xgboost | logloss | 1.7307 | 3.52 | |
| 14_LightGBM_KMeansFeatures | LightGBM | logloss | 1.50112 | 3.24 | |
| 15_LightGBM | LightGBM | logloss | 1.50112 | 1.93 | |
| 16_LightGBM | LightGBM | logloss | 1.68199 | 1.33 | |
| 17_Xgboost | Xgboost | logloss | 1.70489 | 1.57 | |
| 18_Xgboost | Xgboost | logloss | 1.71091 | 1.45 | |
| 19_DecisionTree | Decision Tree | logloss | 2.1686 | 1.15 | |
| 20_LightGBM | LightGBM | logloss | 1.41144 | 1.65 | |
| 21_LightGBM | LightGBM | logloss | 1.41144 | 1.65 | |
| 22_LightGBM | LightGBM | logloss | 1.4546 | 1.55 | |
| the best | Ensemble | Ensemble | logloss | 1.32742 | 1.65 |
AutoML Performance
AutoML Performance Boxplot
Spearman Correlation of Models
Summary of 14_LightGBM
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 15
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 50
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
1.0 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.333333 | 0.5 | 0 | 0 | 0.5 | 0 | 0.333333 | 0 | 0.222222 | 0.208333 | 0.222222 | 1.68199 |
| recall | 0.25 | 0.5 | 0 | 0 | 0.5 | 0 | 0.5 | 0 | 0.222222 | 0.21875 | 0.222222 | 1.68199 |
| f1-score | 0.285714 | 0.5 | 0 | 0 | 0.5 | 0 | 0.4 | 0 | 0.222222 | 0.210714 | 0.219048 | 1.68199 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.222222 | 18 | 18 | 1.68199 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 14_LightGBM_KMeansFeatures
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 15
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 50
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
2.3 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.333333 | 0 | 0.5 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.479167 | 0.537037 | 1.50112 |
| recall | 0.25 | 0.5 | 0 | 1 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.46875 | 0.444444 | 1.50112 |
| f1-score | 0.4 | 0.4 | 0 | 0.666667 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.433333 | 0.42963 | 1.50112 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.444444 | 18 | 18 | 1.50112 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 2 | 0 | 0 | 0 | 1 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 15_LightGBM
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 31
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 50
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
1.0 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.333333 | 0 | 0.5 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.479167 | 0.537037 | 1.50112 |
| recall | 0.25 | 0.5 | 0 | 1 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.46875 | 0.444444 | 1.50112 |
| f1-score | 0.4 | 0.4 | 0 | 0.666667 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.433333 | 0.42963 | 1.50112 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.444444 | 18 | 18 | 1.50112 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 2 | 0 | 0 | 0 | 1 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 16_LightGBM
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 31
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 50
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.5 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.333333 | 0.5 | 0 | 0 | 0.5 | 0 | 0.333333 | 0 | 0.222222 | 0.208333 | 0.222222 | 1.68199 |
| recall | 0.25 | 0.5 | 0 | 0 | 0.5 | 0 | 0.5 | 0 | 0.222222 | 0.21875 | 0.222222 | 1.68199 |
| f1-score | 0.285714 | 0.5 | 0 | 0 | 0.5 | 0 | 0.4 | 0 | 0.222222 | 0.210714 | 0.219048 | 1.68199 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.222222 | 18 | 18 | 1.68199 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 1 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 17_Xgboost
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: multi:softprob
- eta: 0.05
- max_depth: 8
- min_child_weight: 5
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: mlogloss
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.8 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.333333 | 0.5 | 0 | 0.5 | 0 | 0.333333 | 1 | 0.388889 | 0.458333 | 0.518519 | 1.70489 |
| recall | 0.25 | 0.5 | 0.5 | 0 | 0.5 | 0 | 0.5 | 1 | 0.388889 | 0.40625 | 0.388889 | 1.70489 |
| f1-score | 0.4 | 0.4 | 0.5 | 0 | 0.5 | 0 | 0.4 | 1 | 0.388889 | 0.4 | 0.4 | 1.70489 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.388889 | 18 | 18 | 1.70489 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 2 | 0 | 1 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 18_Xgboost
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: multi:softprob
- eta: 0.1
- max_depth: 8
- min_child_weight: 5
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: mlogloss
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.7 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.333333 | 0.5 | 0 | 0.5 | 0 | 0.25 | 1 | 0.388889 | 0.447917 | 0.509259 | 1.71091 |
| recall | 0.25 | 0.5 | 0.5 | 0 | 0.5 | 0 | 0.5 | 1 | 0.388889 | 0.40625 | 0.388889 | 1.71091 |
| f1-score | 0.4 | 0.4 | 0.5 | 0 | 0.5 | 0 | 0.333333 | 1 | 0.388889 | 0.391667 | 0.392593 | 1.71091 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.388889 | 18 | 18 | 1.71091 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 2 | 0 | 1 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 19_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: entropy
- max_depth: 3
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.5 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0 | 0.5 | 0 | 0 | 0.5 | 0.4 | 1 | 0.4 | 0.444444 | 0.35 | 0.311111 | 2.1686 |
| recall | 0 | 0.5 | 0 | 0 | 0.5 | 1 | 1 | 1 | 0.444444 | 0.5 | 0.444444 | 2.1686 |
| f1-score | 0 | 0.5 | 0 | 0 | 0.5 | 0.571429 | 1 | 0.571429 | 0.444444 | 0.392857 | 0.349206 | 2.1686 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.444444 | 18 | 18 | 2.1686 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 2 |
| Labeled as BA | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 1_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: gini
- max_depth: 3
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.8 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.4 | 0.5 | 0 | 0 | 1 | 1 | 0.333333 | 0 | 0.388889 | 0.404167 | 0.403704 | 1.72988 |
| recall | 0.5 | 0.5 | 0 | 0 | 0.5 | 0.5 | 1 | 0 | 0.388889 | 0.375 | 0.388889 | 1.72988 |
| f1-score | 0.444444 | 0.5 | 0 | 0 | 0.666667 | 0.666667 | 0.5 | 0 | 0.388889 | 0.347222 | 0.358025 | 1.72988 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.388889 | 18 | 18 | 1.72988 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 1 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 |
| Labeled as CC | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 |
| Labeled as Fail | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 20_LightGBM
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 15
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 30
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.9 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.5 | 0.25 | 0.5 | 0.666667 | 0 | 0 | 0.333333 | 1 | 0.444444 | 0.40625 | 0.416667 | 1.41144 |
| recall | 0.25 | 0.5 | 0.5 | 1 | 0 | 0 | 0.5 | 1 | 0.444444 | 0.46875 | 0.444444 | 1.41144 |
| f1-score | 0.333333 | 0.333333 | 0.5 | 0.8 | 0 | 0 | 0.4 | 1 | 0.444444 | 0.420833 | 0.411111 | 1.41144 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.444444 | 18 | 18 | 1.41144 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BA | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
| Labeled as CB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 21_LightGBM
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 31
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 30
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.9 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.5 | 0.25 | 0.5 | 0.666667 | 0 | 0 | 0.333333 | 1 | 0.444444 | 0.40625 | 0.416667 | 1.41144 |
| recall | 0.25 | 0.5 | 0.5 | 1 | 0 | 0 | 0.5 | 1 | 0.444444 | 0.46875 | 0.444444 | 1.41144 |
| f1-score | 0.333333 | 0.333333 | 0.5 | 0.8 | 0 | 0 | 0.4 | 1 | 0.444444 | 0.420833 | 0.411111 | 1.41144 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.444444 | 18 | 18 | 1.41144 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BA | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
| Labeled as CB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 22_LightGBM
LightGBM
- n_jobs: -1
- objective: multiclass
- num_leaves: 15
- learning_rate: 0.05
- feature_fraction: 0.8
- bagging_fraction: 0.5
- min_data_in_leaf: 30
- metric: multi_logloss
- custom_eval_metric_name: None
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.8 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.5 | 0.25 | 1 | 0.4 | 0 | 0.5 | 0.5 | 1 | 0.5 | 0.51875 | 0.516667 | 1.4546 |
| recall | 0.25 | 0.5 | 0.5 | 1 | 0 | 0.5 | 0.5 | 1 | 0.5 | 0.53125 | 0.5 | 1.4546 |
| f1-score | 0.333333 | 0.333333 | 0.666667 | 0.571429 | 0 | 0.5 | 0.5 | 1 | 0.5 | 0.488095 | 0.470899 | 1.4546 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.5 | 18 | 18 | 1.4546 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BA | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 2_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: gini
- max_depth: 4
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.7 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.2 | 0 | 0 | 0 | 1 | 0.166667 | 0 | 0 | 0.166667 | 0.170833 | 0.174074 | 4.04011 |
| recall | 0.25 | 0 | 0 | 0 | 0.5 | 0.5 | 0 | 0 | 0.166667 | 0.15625 | 0.166667 | 4.04011 |
| f1-score | 0.222222 | 0 | 0 | 0 | 0.666667 | 0.25 | 0 | 0 | 0.166667 | 0.142361 | 0.151235 | 4.04011 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.166667 | 18 | 18 | 4.04011 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 |
| Labeled as BA | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Labeled as Fail | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 3_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: gini
- max_depth: 4
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
0.7 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 0.2 | 0 | 0 | 0 | 1 | 0.166667 | 0 | 0 | 0.166667 | 0.170833 | 0.174074 | 4.04011 |
| recall | 0.25 | 0 | 0 | 0 | 0.5 | 0.5 | 0 | 0 | 0.166667 | 0.15625 | 0.166667 | 4.04011 |
| f1-score | 0.222222 | 0 | 0 | 0 | 0.666667 | 0.25 | 0 | 0 | 0.166667 | 0.142361 | 0.151235 | 4.04011 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.166667 | 18 | 18 | 4.04011 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 1 | 0 | 1 | 0 | 0 | 0 | 1 |
| Labeled as BA | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 1 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 |
| Labeled as Fail | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 4_Linear
Logistic Regression (Linear)
- n_jobs: -1
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
18.4 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0 | 0 | 0.2 | 0 | 0 | 0.5 | 1 | 0.333333 | 0.3375 | 0.411111 | 1.96024 |
| recall | 0.25 | 0 | 0 | 0.5 | 0 | 0 | 1 | 1 | 0.333333 | 0.34375 | 0.333333 | 1.96024 |
| f1-score | 0.4 | 0 | 0 | 0.285714 | 0 | 0 | 0.666667 | 1 | 0.333333 | 0.294048 | 0.30582 | 1.96024 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.333333 | 18 | 18 | 1.96024 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 1 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as BA | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as BB | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 0 |
| Labeled as CB | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 5_Xgboost
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: multi:softprob
- eta: 0.075
- max_depth: 8
- min_child_weight: 5
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: mlogloss
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
1.4 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.333333 | 0.5 | 0 | 0.5 | 0 | 0.333333 | 1 | 0.388889 | 0.458333 | 0.518519 | 1.70127 |
| recall | 0.25 | 0.5 | 0.5 | 0 | 0.5 | 0 | 0.5 | 1 | 0.388889 | 0.40625 | 0.388889 | 1.70127 |
| f1-score | 0.4 | 0.4 | 0.5 | 0 | 0.5 | 0 | 0.4 | 1 | 0.388889 | 0.4 | 0.4 | 1.70127 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.388889 | 18 | 18 | 1.70127 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 2 | 0 | 1 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of 5_Xgboost_categorical_mix
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: multi:softprob
- eta: 0.075
- max_depth: 8
- min_child_weight: 5
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: mlogloss
- num_class: 8
- explain_level: 0
Validation
- validation_type: split
- train_ratio: 0.9
- shuffle: True
- stratify: True
Optimized metric
logloss
Training time
1.5 seconds
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.333333 | 0 | 0 | 0.5 | 0 | 0.333333 | 1 | 0.333333 | 0.395833 | 0.462963 | 1.7307 |
| recall | 0.25 | 0.5 | 0 | 0 | 0.5 | 0 | 0.5 | 1 | 0.333333 | 0.34375 | 0.333333 | 1.7307 |
| f1-score | 0.4 | 0.4 | 0 | 0 | 0.5 | 0 | 0.4 | 1 | 0.333333 | 0.3375 | 0.344444 | 1.7307 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.333333 | 18 | 18 | 1.7307 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 2 | 0 | 1 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CB | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as CC | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
Summary of Ensemble
Ensemble structure
| Model | Weight |
|---|---|
| 14_LightGBM | 1 |
| 19_DecisionTree | 2 |
| 1_DecisionTree | 3 |
| 20_LightGBM | 10 |
Metric details
| AA | BA | BB | CB | CC | DC | DD | Fail | accuracy | macro avg | weighted avg | logloss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| precision | 1 | 0.25 | 0 | 1 | 0.5 | 0 | 0.333333 | 0.666667 | 0.444444 | 0.46875 | 0.527778 | 1.32742 |
| recall | 0.25 | 0.5 | 0 | 1 | 0.5 | 0 | 0.5 | 1 | 0.444444 | 0.46875 | 0.444444 | 1.32742 |
| f1-score | 0.4 | 0.333333 | 0 | 1 | 0.5 | 0 | 0.4 | 0.8 | 0.444444 | 0.429167 | 0.425926 | 1.32742 |
| support | 4 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 0.444444 | 18 | 18 | 1.32742 |
Confusion matrix
| Predicted as AA | Predicted as BA | Predicted as BB | Predicted as CB | Predicted as CC | Predicted as DC | Predicted as DD | Predicted as Fail | |
|---|---|---|---|---|---|---|---|---|
| Labeled as AA | 1 | 3 | 0 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BA | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 |
| Labeled as BB | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
| Labeled as CB | 0 | 0 | 0 | 2 | 0 | 0 | 0 | 0 |
| Labeled as CC | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 |
| Labeled as DC | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 |
| Labeled as DD | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 |
| Labeled as Fail | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Precision Recall Curve
